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Abstract 

Realizing  the  benefits  of  network-centric  warfare — in  terms  of  improved  access  to  high-quality 
information,  speed  of  command,  and  dominant  application  of  forces — will  require  a  synergy 
among  three  dimensions  of  change:  Technology,  force  organization,  and  team  processes.  To 
achieve  the  potential  advantages  of  new  technological  capabilities  requires  that  we  reexamine  old 
rules  of  business  and  force-structures  and  recast  them  in  ways  that  allow  for  increased  flexibility 
and  application  of  force  where  and  when  it  is  required.  Experiments  and  exercises  designed  to 
explore  alternative  structures,  processes,  and  the  impacts  of  information  technologies  are 
complex,  precisely  because  they  force  change  in  all  three  dimensions.  The  challenge  of 
assessing  the  impacts  of  these  changes  in  terms  of  individual,  team,  and  overall  organizational 
performance,  are  great.  This  paper  describes  an  approach  to  dealing  with  the  complexity  of 
assessment  described  above  through  the  application  of  “bridge”  experiments  that  start  with  a 
blend  of  modeling  and  experimentation  in  the  laboratory — to  thoroughly  explore  core  concepts 
and  test  new  assessment  ideas  under  controlled  conditions — and  scale  to  meet  the  challenges  of 
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field-level  performance  assessment  by  emphasizing  those  issues  that  proved  to  be  performance 
drivers  in  the  laboratory. 


1.  Introduction 

Realizing  the  benefits  of  network-centric  warfare — in  terms  of  improved  access  to  high-quality 
information,  speed  of  command,  and  dominant  application  of  forces — will  require  a  synergy 
among  three  dimensions  of  change:  Technology,  force  organization,  and  team  processes.  To 
achieve  the  potential  advantages  of  new  technological  capabilities  requires  that  we  reexamine  old 
rules  of  business  and  force-structures  and  recast  them  in  ways  that  allow  for  increased  flexibility 
and  application  of  force  where  and  when  it  is  required.  Experiments  and  exercises  designed  to 
explore  alternative  structures,  processes,  and  the  impacts  of  information  technologies  are 
complex,  precisely  because  they  force  change  in  all  three  dimensions.  The  challenge  of 
assessing  the  impacts  of  these  changes  in  terms  of  individual,  team,  and  overall  organizational 
performance,  are  great. 

This  paper  describes  an  approach  to  dealing  with  the  complexity  of  assessment  described  above 
through  the  application  of  “bridge”  experiments  that  start  with  a  blend  of  modeling  and 
experimentation  in  the  laboratory — to  thoroughly  explore  core  concepts  and  test  new  assessment 
ideas  under  controlled  conditions — and  scale  to  meet  the  challenges  of  field-level  performance 
assessment  by  emphasizing  those  issues  that  proved  to  be  performance  drivers  in  the  laboratory. 
The  remainder  of  this  paper  discusses  an  application  of  this  approach  in  preparation  for  the 
Global  Wargame,  held  at  the  Naval  War  College,  in  August  of  1999. 

2.  Bridge  to  Global 

Three  months  prior  to  the  Global  ’99  Wargame,  a  team  of  warfighters,  under  the  guidance  of 
CCG1  and  collaborating  researchers  from  the  Adaptive  Architectures  for  Command  and  Control 
(A2C2)  project  sponsored  by  the  Office  of  Naval  Research,  conducted  a  training  exercise  at  the 
Naval  Postgraduate  School  entitled  “Bridge  to  Global  ’99.”  Some  of  the  objectives  met  by  the 
Bridge  to  Global  (BTG)  exercise  were  to  provide  training  and  exposure  to  alternative  Joint  Task 
Force  (JTF)  command  and  control  (C2)  architectures;  to  observe  the  impacts  of  alternate  JTF  C2 
architectures  on  command  processes;  to  explore  network-centric  warfare  (NCW)  concepts  and 
technologies  in  a  controlled  setting;  to  validate  and  refine  A2C2  organizational  design 
methodologies  in  operational  settings  with  operational  experts;  and  to  explore  the  capabilities  of 
new  information  technology  (IT21)  tools  as  support  for  C2  information  management  and 
decision  making. 

BTG  provided  a  low-cost,  feedback-rich  environment  in  which  to  explore  NCW  and 
organizational  innovations  with  operational  experts,  and  to  collect,  summarize,  and  feedback 
data  and  observations  to  maximize  the  training  impacts  and  lessons  learned.  As  a  result,  BTG 
provided  reliable  insights  concerning  the  efficiency  of  new  organizational  changes  and 
supporting  technologies,  many  of  which  found  their  way  into  the  Global  ’99  game. 

Additionally,  BTG  provided  the  A2C2  research  team  with  a  forum  to  explore  new  assessment 
and  performance  measurement  techniques  in  a  rich  context.  Fessons  learned  in  this  area  have 


helped  the  A2C2  team  scale  many  of  their  laboratory-measurement  and  data-analysis  methods  to 
the  broader  and  more  chaotic  field  setting  of  GLOBAL  ’99. 

During  BTG,  the  A2C2  research  team  was  able  to  employ  a  number  of  observational  and  survey 
data  collection  methods  to  capture  qualitative  and  quantitative  performance  data.  As  the 
command  staff  was  observed  playing  the  simulated  wargame,  the  observation  team  collected 
communications  data,  workload  metrics,  subjective  and  objective  data  about  the  tools  used  and 
the  methods  tried  to  make  the  best  use  of  available  technologies,  and  subjective  data  about  the 
organizational  impacts  of  these  technologies.  At  the  end  of  each  day,  the  observation  team 
briefed  game  players  on  the  days  results,  and  at  the  beginning  of  the  next  day,  decisions  were 
made  about  ways  to  change  organizational  processes  to  improve  performance.  The  results  of  this 
process,  after  a  week,  were  a  number  of  insights  and  “rules  of  business”  that  were  taken  to 
Global  Wargame. 

2.1  Methods  Used  at  BTG 

The  decision  regarding  appropriate  data  collection  methods  at  BTG  was  a  pragmatic  one — the 
assessment  team  collected  data  that  could  be  quickly  turned  around  as  feedback  to  improve  the 
team’s  communication  processes  and  overall  performance  while  remaining  non-intrusive  and 
requiring  minimal  participation  by  the  players.  Following  are  some  of  the  data  collection 
techniques  used. 

2.1.1  Communication/Coordination 

To  study  the  team’s  communication  patterns  and  ability  to  coordinate,  two  members  of  the 
assessment  team  used  handheld  computers  with  touch  sensitive  screens  to  record  the  type  of 
information  transferred  from  one  group  within  the  team  to  another.  By  using  a  Java  applet 
program,  the  observers  were  able  to  collect  time- stamped  data  of  team  interaction  simply  by 
tapping  on  a  grid  (see  Figure  1)  that  specified  type  of  information  exchanged  (i.e.,  request  for 
information,  request  for  action  or  task,  request  for  resource  utilization,  request  for  change, 
transfer  of  information,  statement  about  performing  action  or  task,  statement  about  using 
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Figure  1 .  an  example  of  a  real-time  data  collection  grid  for  team 

communications 

resources,  statement  about  changing,  and  any  information  technology  (IT21)  tool  use)  and  the 
source/destination  dyad  of  that  exchange  (the  groups  within  the  team  were  called  Flag  (F),  Alpha 


(A),  Bravo  (B),  and  Charlie  (C);  in  the  grid,  “F/A”  signifies  that  the  information  went  from  the 
Flag  cell  to  cell  Alpha). 

In  addition  to  collecting  the  time- series  data  the  data  collection  grid  afforded,  other  members  of 
the  assessment  team  kept  detailed  notes  about  specific  communications  from  one  cell  to  another. 
These  observers  recorded  the  communications  that  took  place  when  important  events  occurred 
with  a  special  focus  on  those  communications  concerning  the  IT21  tools,  the  organizational 
structure,  and  team  processes. 

2.1.2  Teamwork  Skills 

Those  same  observers  keeping  detailed  notes  on  specific  communication  interchanges  also  noted 
specific  teamwork  behaviors  such  as  the  team’s  coordination  behavior  (e.g.,  what  were  the 
decision  processes  used  to  solve  problems  when  they  arise,  what  problems  did  the  team 
encounter  in  solving  the  problem,  what  were  the  positive  aspects  of  their  coordination  behavior) 
and  IT21  tool  use  (e.g.,  what  problems  did  they  encounter  and  what  protocol  developed  for  the 
tools  use).  We  also  had  the  observers  and  each  team  member  fill  out  a  teamwork  assessment 
questionnaire  that  measured  perceived  team  performance  in  communication  behavior, 
monitoring  behavior,  back-up  (i.e.,  supporting)  behavior,  coordinating  behavior,  and  team 
orientation  (e.g.,  placing  team  goals  ahead  of  personal  goals). 

2.1.3  Workload 

To  measure  workload,  we  used  a  revised  version  of  the  six  item  task  load  index  (TLX)  workload 
questionnaire  that  measures  each  team  member’s  perceived  task  load  in  terms  of  mental  demand, 
physical  demand,  temporal  demand,  performance,  effort,  and  frustration.  In  the  revised  version, 
we  did  not  measure  physical  demand  or  performance,  and  we  gathered  information  for  self 
workload,  cell  workload,  and  perceived  workload  for  the  other  cells. 

2.1.4  Situational  Awareness 

To  determine  situation  awareness  (SA)  we  used  a  self  report  instrument  for  which  the 
participants  are  asked  to  remember  a  specific  salient  event  and  describe  what  they  and  the  other 
team  members  were  doing  when  that  event  occurred. 

2.2  BTG  Results 

The  above  data  collection  techniques  proved  to  be  quite  diagnostic  regarding  many  facets  of  the 
team’s  performance  -  from  the  effects  of  diverse  architectures  to  team  information  management. 
Some  of  the  most  relevant  findings  are  described  below. 


2.2. 1  Diverse  Architectures 


Before  attending  BTG,  members  of  the  A2C2  team  used  mathematical  modeling  to  generate 
architectures  that  optimize  speed  of  command  -  the  JTF  organization  must  meet  mission 
objectives  at  maximum  speed  of  command  or  op  tempo,  acceptable  workload  -  workload  should 
be  balanced  across  cells,  effective  team  coordination  -  inter-cell  coordination  requirements 
should  be  minimal,  and  supported-supporting  relationships  -  the  architecture  should  support 
inter-cell  interaction.  The  optimized  architectures  for  the  two  different  phases  of  the  BTG 
scenario  are  shown  in  figure  2.  It  was  not  surprising  to  find  that  the  participants  were  generally 
pleased  with  both  architectures,  rating  them  high  on  the  effectiveness  of  that  architecture  to  help 
them  complete  their  mission 
tasks,  perform  alternative 
strategies  and  maintain  effective 
communications  and 

coordination;  what  was  surprising 
was  the  finding  that  these 
“optimal”  architectures  had  some 
detrimental  effects:  as  the 

organizational  structure  reduces 
the  need  for  coordination,  the 
individuals  within  that 
organization  began  to  lose  sight  of 
the  common  operational  picture 
(COP).  It  was  found  that  regular 
e-briefings  by  each  team  to  the 
entire  JTF  was  needed  to  maintain 
global  SA. 

2.2.2  Adaptation 

As  is  stated  in  Joint  Vision  2010,  it  is  imperative  that  organizations  be  flexible  enough  to  adapt 
to  dynamic  and  diverse  mission  demands.  During  BTG  the  research  team  monitored  for  such 
adaptation  -  focusing  on  what  enabled  or  hindered  the  necessary  flexibility  -  and  determined  that 
there  are  a  number  of  factors  that  promote  adaptation  when  present  and  hinder  adaptation  when 
absent.  Some  of  the  most  important  enablers  were  the  commander’ s  intent  and  clarity  of  roles, 
optimized  C2  architecture,  and  real-time  communications  and  shared  awareness.  As  long  as  the 
participants  knew  what  was  expected  of  them  in  the  mission,  the  C2  architecture  was  optimized 
to  meet  mission  demands,  and  the  nodes  within  the  team  were  able  to  communicate  in  real-time 
to  keep  everyone  alert  and  responsive  to  the  present  situation,  the  team  was  able  to  adapt  to  the 
ever  changing  situation;  if  any  of  these  enablers  were  missing  the  team  was  incapable  of  the 
flexibility  necessary  to  be  optimally  effective. 

From  the  assessment  team  observations,  it  was  further  concluded  the  presence  of  enablers  is 
necessary  but  not  sufficient  to  ensure  successful  adaptability;  the  team  has  to  take  a  more 
proactive  role  and  begin  the  adaptation  process  before  it  is  needed.  If  the  team  waits  until 
change  is  absolutely  necessary  it  is  unlikely  that  they  will  be  able  to  adapt  in  a  timely  manner. 
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Figure  2.  JTF  C2  Architectures  at  BTG 


Understanding  the  precursors  to  adaptation  is  in  need  of  further  study.  Although  we  were  not 
able  to  ascertain  what  specific  predictors  of  the  need  for  change  were  (e.g.,  an  abrupt  change  in 
the  comm,  traffic),  we  were  able  to  conclude  that  the  team  needs  to  remain  constantly  aware  of 
future  likely  points  of  failure,  the  appropriateness  of  the  distribution  of  personnel  and  skills  for 
current  and  future  tasks,  and  the  appropriateness  of  the  asset/resource  matches  for  current  and 
future  tasks. 

2.2.3  Team  Coordination  and  Maintaining  Situation  Awareness 

One  requisite  for  effective  team  performance  is  shared  situation  awareness  (SA).  With  this  in 
mind,  the  participants  of  BTG  were  asked  to  rate  a  number  of  factors  on  their  importance  for 
team  coordination  and  maintaining  a  shared  situation  awareness,  and  the  consensus  was  that  real 
time  communications  were  the  most  important  contributor  for  shared  SA  (as  one  of  the 
participants  wrote  on  the  survey,  “Everything  else  is  built  on  comms.”).  Preplanning  and  having 
a  common  operational  procedure  were  important  also,  but  real  time  communications  was 
consistently  rated  the  highest. 

At  BTG,  the  only  true  real-time  communication  were  the  voice  comms,  none  of  the  others  (e.g., 
chat,  email,  whiteboard)  required  the  constant  attention  that  helped  the  participants  maintain  a 
shared  SA.  Other  than  real-time  communications,  some  enablers  of  team  coordination  and 
shared  SA  are  tool  practice — participants  needed  ample  opportunity  to  use  the  tools  in  the 
contest  of  the  wargame  before  they  could  achieve  the  tools’  full  potential,  model-based 
architecture — the  organizational  structure  specially  modeled  to  enable  team  coordination  was  a 
key  component  of  the  team’s  success,  and  multiple  IT21  tools — no  one  tool  would  have  been 
sufficient  for  all  of  the  coordination  and  situation  awareness  needs.  This  last  point  is  explained 
in  more  detail  in  the  next  section. 

2.2.4  NCW  IT  tools 

In  BTG  participants  were  dropped  into  the  context  of  a  complex  wargame  simulation  and 
provided  with  a  range  of  digital  tools  for  use  in  support  of  team  awareness,  communication,  and 
coordination,  and  collaboration.  At  the  start  of  BTG,  the  core  command  team  was  provided  with 
very  little  in  the  way  of  guidelines  for  employment  of  the  new  tools.  Instead,  protocols  and 
patterns  of  effective  usage  were  allowed  to  evolve.  Specifically,  the  BTG  organization  was 
given  a  rich  set  of  communication  and  collaboration  tools  that  included  audio  communications, 
real-time  chat,  email,  shared  whiteboard,  shared  map  display,  and  videoconferencing  (see  Figure 
3  for  a  screen  shot  of  the  IT21  tools  used).  Each  of  these  tools  was  trained  in  isolation  until  team 
members  understood  the  mechanics  of  use,  but  no  specific  guidelines  were  given  about  how  the 
team  should  use  the  tools  to  do  their  job. 
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Figure  3.  A  screen  shot  of  the  IT21  tools. 


Over  the  course  of  one  week  of  game  play,  the  command  team  evolved  a  number  of  patterns  of 
technology  use  that  worked  very  well  for  the  small  team  of  thirty  five  players,  and  was  well 
suited  to  the  dynamics  of  the  scenarios  played  in  the  game  (which  were  modeled  after  the 
scenarios  to  be  played  during  the  larger  Global  Wargame  several  months  later).  The  emergent 
protocols,  which  can  be  found  in  Figure  4,  suggest  that  effective  communication  exists  only 
when  the  purpose  of  the  communication  is  matched  to  the  type  of  media.  For  example,  if  the 
communication  is  vital  and  must  be  attended  to  immediately,  then  the  tool  used  must  be  voice 
comms  because  that  is  the  only  communication  venue  available  that  fosters  such  immediate 
attention;  if  a  large  file  had  to  be  sent  as  an  attachment,  email  was  used.  It  is  important  to  note 
that  the  whiteboard  appeared  to  have  great  potential  in  planning  and  maintaining  shared  SA; 
however,  the  participants  were  least  familiar  with  this  tool  and  chose  to  use  the  other,  more 
familiar  tools,  instead. 
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Figure  4.  IT21  Tools:  Evolved  Protocol  and  Perceived  Importance 

3.  Crossing  the  Bridge  to  Global  Wargame 

Based  on  the  results  of  Bridge  to  Global,  the  A2C2  assessment  team  went  to  Global  Wargame 
’99  with  custom  tailored  measures  and  a  focus  inspired  by  our  experimental  findings.  We 
tailored  our  data  collection  and  assessment  efforts  to  capture  impacts  of  information  technologies 
and  collaborative  tools  (e.g.  chat,  shared  COP)  on  team  process  and  performance,  and  looked 
closely  at  efforts  made  by  the  JTF  and  its  subordinate  cells  to  leverage  these  tools  to  accomplish 
improved  speed  of  command  through  an  Effects-Based  approach  to  Command  and  Control  (C2). 

3.1  Methods  Used  at  Global 

To  provide  adequate  assessment  coverage,  a  handful  of  raters  from  the  A2C2  assessment  team 
attended  GLOBAL  ’99  where  they  used  very  similar  collection  techniques  as  those  used  for  BTG 
to  gather  information  on  teamwork  skills,  workload,  and  situational  awareness;  however,  because 
there  were  so  many  more  players  spread  out  over  a  much  larger  area,  the  data  collection 
technique  for  Communication/Coordination  had  to  be  revised.  Instead  of  relying  on  the 
handheld  computers  to  capture  every  information  transaction,  each  member  of  the  assessment 
team  was  equipped  with  rating  manuals  specifically  designed  to  capture  interactions  between 
team  members,  team  coordination  behaviors,  adaptations  of  the  team  architecture  over  time, 
changes  in  command  structure,  and  uses  of  IT21  tools  (see  Figure  5  for  examples  of  pages  from 
the  raters’  manuals).  Each  rater — standing  in  one  of  the  following  seven  areas:  Current 
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Figure  5.  Rating  Communication  and  Coordination  in  Global 
Operations,  Future  Operations,  Effects  Coordination  Board,  Network  Operations  Center, 
Subordinate  Joint  Task  Force  (SJTF)  A,  SJTF  B,  SJTF  C — compiled  extensive  observations  on 
the  team  interactions. 


3.2  Lessons  Learned 


Four  broad  themes,  or  lessons  learned,  emerged  from  our  observations,  and  these  will  be  used  to 
provide  a  framework  to  organize  this  final  discussion.  These  themes  were: 

•  The  importance  of  training — While  the  BTG  was  a  research  experience  for  the  A2C2 
team,  it  was  a  valuable  training  experience  for  the  command  staff.  Without  adequate,  in¬ 
context  training,  it  is  not  possible  for  an  organization  to  realize  the  promises  of  new 
technologies. 

•  New  tools  require  new  rules — Just  as  there  are  certain  kinds  of  meetings  that  just  have  to 
be  done  face  to  face,  there  are  certain  kinds  of  messages  that  are  better  sent  as  email  than 
as  a  chat  message.  Discovering,  codifying,  and  adhering  to  rules  and  heuristics  for  tools 
use  can  make  it  much  easier  for  an  organization  to  take  advantage  of  these  tools. 

•  The  myth  of  “global  situation  awareness” — The  global  availability  of  information  can 
create  an  illusion  that  everyone  knows  what  you  know,  and  lead  to  errors  of  omission  or 
false  expectations.  It  is  important  that  tools,  plans,  and  other  organizational  methods  be 
used  to  insure  that  team  members  develop  adequate  understanding  of  one  another  roles, 
information  purview  and  information  requirements. 


•  Knowledge  management  may  be  a  distinct  organizational  role — Particularly  when  the 
size  of  an  organization  gets  large  (as  in  the  Global  Wargame)  the  job  of  managing  all 
potential  sources  of  information,  and  making  sure  decision  makers  have  access  to  what 
they  need  when  they  need  it,  requires  special  expertise.  It  is  increasingly  recognized  that 
the  role  of  “knowledge  manager”  is  an  important  new  organizational  role. 

In  the  paragraphs  that  follow,  we  discuss  these  themes  in  greater  depth,  using  examples  from 
BTG  and  Global  wargame. 

3.2.1  The  Importance  of  Training 

Changes  in  the  availability  of  information  and  abilities  of  individuals  to  share  what  they  know 
across  traditional  boundaries  alter  organizational  dynamics  in  a  number  of  ways  (Prestridge, 
Hansen,  Sessa,  &  Kossler,  1999).  To  be  successful,  technological  change  must  be  accompanied 
by  other  changes  such  as  training,  structural  changes  in  the  organization,  and  changes  in  the 
processes  and  rules  of  business  communication  (Offermann  &  Eller,  1999;  MacMillan,  Paley, 
Levchuk,  &  Serfaty,  1999). 

The  transition  from  BTG — where  players  were  afforded  individualized  tool  training  with  ample 
feedback — to  Global  Wargame — where  many  of  the  players  were  exposed  to  network-based 
communication  and  collaboration  tools  for  the  first  time — made  one  thing  very  clear:  Quality 
digital  skills  training,  embedded  in  the  context  in  which  the  tools  will  ultimately  be  used,  is  a 
crucial  determinant  of  an  organization’s  success  in  practice.  While  it  is  not  surprising  to  say  that 
training  and  practice  lead  to  improved  performance,  the  point  goes  deeper  than  that.  What  BTG 
provided  was  more  than  an  opportunity  to  work  hands-on  with  the  tools:  It  additionally  provided 
players  with  a  cost-free,  feedback  rich,  environment  in  which  to  explore  different  ways  that  the 
tools  might  be  used  to  practice  C2. 

There  were  two  distinct  types  of  training  that  took  place  at  BTG.  The  first  was  what  is  often 
called  “buttonology”  training,  or  training  in  the  mechanics  of  tool  use.  The  second,  and  arguably 
more  important,  type  of  training  was  the  in-context  training,  where  new  tools  were  truly  tested 
and  experienced  in  the  context  of  job  performance.  In  the  transition  from  BTG  to  Global 
Wargame,  the  assessment  team  was  afforded  a  glimpse  at  the  ramifications  of  providing  one  type 
of  training  without  the  other. 

At  BTG,  players  received  hands-on  buttonology  training  early  in  the  week,  to  familiarize  them 
with  the  mechanics  of  the  tools,  their  capabilities,  and  a  few  suggestions  about  how  they  might 
be  employed.  Players  started  chat  groups,  joined  and  left  them,  set  up  email  addresses,  tested 
their  video  connections,  and  interacted  with  the  white  board  and  shared  map  displays.  After  they 
were  familiar  with  the  tools,  the  simulation  game  started,  and  players  were  faced  with  the  task  of 
figuring  out  how  to  use  all  of  these  new  capabilities  to  help  them  work.  This  “free-form”  phase 
was  training  in  the  sense  that,  though  they  were  left  to  their  own  sense  of  how  to  proceed,  the 
observation  team  was  present  to  collect  data  on  their  progress  and  choices,  and  summarized  this 
material  as  structured  feedback  and  a  framework  for  self  reflection.  This  provided  the  command 
staff  the  ability  to  chart  their  own  progress  with  the  tools  and  actively  participate  in  the 


development  of  rules  of  conduct  and  best  practice  for  tool  use.  The  products  of  this  effort  are 
discussed  in  the  following  section. 

When  the  BTG  group  went  to  Global  Wargame,  they  were  some  of  the  only  players  who  had 
been  afforded  opportunities  to  work  with  new  technologies  in  the  context  of  their  work,  and  it 
showed.  As  is  too  often  the  case,  the  Global  Wargame  developers  scheduled  time  for  tool 
training  early  in  the  week,  but  focused  exclusively  on  butonology,  with  no  time  for  structured, 
in-context  tool  training.  As  you  might  expect,  four  hundred  people,  sharing  time  on  fewer  than 
four  hundred  computer  screens,  in  an  attempt  to  learn  how  to  use  an  arsenal  of  new  tools  to  help 
them  do  their  job,  was  less  than  perfectly  effective.  As  the  game  designers  quickly  learned, 
valuable  game  time  ended  up  being  used  as  training  time,  during  which  players  really  educated 
themselves  about  the  ways  new  technologies  can  be  used  to  help  them  conduct  C2. 

As  will  become  evident  in  the  following  sections,  the  training  required  to  blend  new  technologies 
with  changes  in  organizational  structure  and  organizational  processes  only  happens  when  the 
buttonology  is  over  and  the  organization  puts  new  technologies  to  the  test,  in  context.  A  major 
lesson  learned  from  the  BTG  and  Global  Wargame  experience  is  that  in-context  tool  use, 
coupled  with  data  collection,  structured  feedback,  and  ample  self  reflection  are  required  to 
develop,  agree  to,  and  codify  organizational  rules  of  practice  that  allow  team  members  to 
fearlessly  utilize  new  tools  in  ways  that  they  are  confident  will  be  consistent  with  their 
teammates  understanding  of  the  models  of  communication  and  collaboration  adopted  by  the 
organization.  These  rules  of  practice  derived  in  the  context  of  C2  have  already  been  discussed 
and  are  further  detailed  in  the  next  section. 

3.2.2  New  Tools  Require  New  Rules 

Just  as  there  are  certain  kinds  of  meetings  that  just  have  to  be  done  face  to  face,  there  are  certain 
kinds  of  messages  that  are  better  sent  as  email  than  as  a  chat  message.  Discovering,  codifying, 
and  adhering  to  rules  and  heuristics  for  tools  use  can  make  it  much  easier  for  an  organization  to 
take  advantage  of  these  tools. 

A  primary  finding  from  our  exploratory  research  has  been  that  the  proliferation  of  new 
communication  and  coordination  supporting  tools  requires  a  brand  new  set  of  rules,  protocols, 
and  guidelines-for-use  to  help  teams  understand  when  and  how  best  to  use  them.  Each  of  the 
tools  mentioned  differs  in  the  time  they  take  to  employ,  the  delays  associated  with  message 
delivery,  bandwidth  requirements,  and  invasiveness  (to  name  a  few). 

To  be  used  effectively,  the  team  required  a  rudimentary  set  of  guidelines  for  how  to  utilize  their 
new  tools  to  get  the  job  done.  Over  the  course  of  four  days  of  mission  practice,  the  team 
developed  a  simple  protocol  (outlined  in  figure  3)  in  which  email  was  restricted  to  cases  in  which 
the  message  to  be  sent  had  multiple  recipients  and  an  attached  file  such  as  a  map  or  target  list. 
Chat  sessions  turned  out  to  be  key  as  a  means  of  providing  a  running  record  of  organizational 
activities,  and  this  tool  evolved  into  a  multipurpose  communication  channel  that  kept  the  radios 
free  for  other  varieties  of  communication.  Video  turned  out  to  be  too  much  trouble  in  the  heat  of 
battle,  but  it  was  suggested  that  this  technology  will  be  crucial  in  planning.  The  whiteboard  was 
rarely  used  even  though  a  number  of  players  recognized  the  potential  usefulness  of  such  a  tool — 


they  cited  their  unfamiliarity  with  the  tool  as  being  an  insurmountable  obstacle  to  its  use  in  the 
fast  paced  wargame  (again,  this  simply  re-emphasizes  the  need  for  proper  training  in  tool  use  and 
protocol).  These  protocols  were  further  tested  and  refined  several  months  later  in  the  larger 
Global  ’99  wargame. 

While  many  organizations  have  long  histories  of  audio  communications  as  a  primary  means  of 
coordinating  distributed  team  members,  the  protocols  and  methods  that  have  grown  up  around 
the  radio  are  not  always  the  best  fit  for  collaboration  supported  by  new,  network-based  tools. 
With  continued  use,  new  rules  are  evolving  for  email,  chat,  videoconferencing,  shared  graphics 
spaces  (i.e.  whiteboards  and  common  map  displays)  and  other  network  communication  tools. 

3.2.3  The  myth  of  “global  situation  awareness ” 

The  ability  to  stay  informed,  and  up-to-date  is  a  promise  that  has  long  been  associated  with 
network-based  computer  services,  particularly  with  the  advent  of  the  internet  and  associated 
technologies  such  as  personalization  and  information  push.  In  military  C2  contexts,  this  promise 
is  often  exaggerated  still  further  to  suggest  that  every  person  will  have  access  to  all  possible 
information,  and  will  therefore  be  constantly  “situationally  aware.”  This  myth  can  be  hazardous 
for  several  reasons:  It  can  lead  the  belief  that  others  know  things  that  they  do  not  and  thus  cause 
individuals  to  develop  expectations  about  behaviors  that  won’t  happen,  it  can  cause  individuals 
to  push  unintelligible  data  to  others,  or  it  can  lead  to  omissions  on  the  assumption  that 
information  available  to  one  is  available  to  all  and  therefore  not  worth  sending.  Regardless  of 
the  outcome,  the  myth  of  global  situation  awareness  can  impact  the  ability  of  the  organization  to 
stay  informed,  coordinate  activities,  and  manage  information. 

The  A2C2  team  observed  several  interesting  themes  related  to  the  myth  of  global  awareness,  but 
two  stand  out  as  particularly  relevant  in  the  context  of  organizational  change:  The  importance  of 
information  management  skills  and  training,  and  the  multi- source  nature  of  what  the  military 
refers  to  as  the  “common  operating  picture.”  These  will  be  discussed  in  turn. 


Information _ management _ and 

organizational _ knowledge 

Understanding  how  to  manage 
information  in  an  organizational  context 
requires  more  than  a  passing  familiarity 
with  the  “cc”  function  in  an  email  client. 
Managing  information  well  requires 
information  filtering,  fusion,  push,  and 
pull,  all  informed  by  knowledge  of 
organizational  roles  and  shared  goals. 
As  can  be  seen  in  Figure  4,  some  of  the 
most  troublesome  barriers  to  proper 
information  access  has  to  do  with  not 
knowing  where  to  get  information  and 
inadequate  responses  to  information 
requests.  With  proper  training  and 
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Figure  6.  Barriers  to  Information  Access 


organizational  knowledge,  these  two  barriers  can  be  overcome. 


Information  Overload  and  the  Common  Operating  Picture  One  of  the  key  findings  of  our 
research  was  that,  with  the  increasing  number  of  information  sources  available  in  the  modern 
work  environment,  no  one  tool  can  any  longer  be  counted  on  as  the  single  source  for  keeping  the 
team  up-to-date  and  coordinated.  The  military  concept  of  a  “common  operating  picture”  may 
not  be  a  picture  in  the  graphic  sense  at  all,  in  fact.  Rather,  the  common  operating  picture  is  a 
sense  of  shared  awareness  fostered  by  the  collection  of  globally  available  information  sources 
and  tools. 

3.2.4  Knowledge  management  may  be  a  distinct  organizational  role 

Particularly  when  the  size  of  an  organization  gets  large  (as  in  the  Global  Wargame)  the  job  of 
managing  all  potential  sources  of  information,  and  making  sure  decision  makers  have  access  to 
what  they  need  when  they  need  it,  requires  special  expertise.  It  is  increasingly  recognized  that 
the  role  of  “knowledge  manager”  is  an  important  new  organizational  role.  Because  of  this,  the 
assessment  team  decided  that  it  may  be  necessary  to  dedicate  an  individual  or  team  of  people  as 
knowledge  managers  whose  sole  job  it  is  to  monitor  multiple  information  sources,  interpret  and 
disseminate  data  and  information  in  support  of  the  team.  In  the  context  of  the  military  wargame 
a  new  cell,  the  Network  Operations  Center  (NOC),  was  developed  to  look  at  communication 
traffic  patterns  and  flow  in  the  IT21  tools  to  optimize  “smart  push  -  smart  pull”  to  make  sure  the 
players  are  not  looking  at  old  information. 

The  NOC  was  implemented  in  Global  ’99  but  was  not  a  successful  knowledge  manager;  instead, 
a  couple  of  players  who  were  extremely  knowledgeable  about  both  the  job  that  needed  to  be 
done,  as  well  as  the  information  networking  operations  being  used,  rose  to  the  occasion  and 
filled  the  knowledge  management  void.  It  appears  as  though  just  knowing  the  information  flow 
and  traffic  patterns  is  not  enough  to  guarantee  knowledge  management,  a  solid  understanding  of 
the  job — or  the  context  in  which  the  knowledge  is  to  be  used — is  needed  for  proper  information 
management.  Next  year,  for  Global  ’00,  one  person  in  every  cell  will  be  thoroughly  trained  in 
both  the  job  that  the  cell  is  responsible  for  as  well  as  in  the  information  technology  being  used. 
This  person  will  be  the  knowledge  manager  for  their  particular  cell.  Similar  positions  would  be 
of  use  in  nonmilitary  organizations  to  make  sure  organizational  members  are  kept  up-to-date 
with  information  relevant  for  their  organizational  position.  This  person  must  be  more  than  a 
systems  administrator  who,  like  the  NOC,  would  know  the  information  network  but  not  the 
specific  jobs;  this  person  must  be  thoroughly  knowledgeable  of  the  job  if  s/he  is  to  successfully 
manage  the  knowledge. 

One  further  interpretation  of  the  above  findings — that  the  knowledge  managers  rose  not  from  the 
network  administrators  whose  job  it  is  to  oversee  all  information  flow,  but  from  players  who  just 
happened  to  have  access  to  all  the  information  because  of  the  omnipresent  information  flow — is 
that  network  communications  tools  can  flatten  an  otherwise  hierarchical  organization  by 
providing  a  conduit  for  rapid  information  dissemination  anywhere  in  the  team.  Thus,  we  found 
that  collaborative  tools  can  have  impacts  at  the  organizational  level  in  ways  that  radio  never  had. 


4.  Conclusion 


Building  adaptive  organizations  is  proving  to  be  quite  a  challenge,  requiring  simultaneous 
improvements  among  the  three  dimensions  of  organizational  structure,  team  process,  and 
information  technology.  The  experiments  and  studies  designed  to  explore  changes  in  these  three 
dimensions  are  quite  complex,  and  span  the  entire  experimental  research  design  arena  from  basic 
research  to  laboratory  experiments  to  field  research  practiced  in  wargames  such  as  Global  ’99. 
Studies  that  test  the  models  developed  in  basic  research  in  such  a  way  that  the  models  can  be 
improved  upon  and  scaled  up  for  larger  wargames  are  essential  if  we  are  to  successfully 
implement  new  concepts  along  all  three  dimensions — BTG  proved  to  be  such  a  study.  Not  only 
did  BTG  allow  for  more  control  than  is  possible  in  the  operational  exercises — allowing  the 
participants  to  get  much  needed  practice  and  feedback,  but  it  provided  more  realistic  situations 
than  the  usual  lab  experiment — allowing  for  the  testing  of  hypotheses,  theories,  and  models  in  a 
more  realistic  setting,  and  it  shortened  tool  familiarization  time  at  Global  so  the  assessment  team 
could  test  and  adapt  NCW  architectures — which  requires  that  the  participants  already  be  trained 
in  them.  “Lesson’s  learned”  from  BTG  and  Global  are  discussed  in  this  paper  and  are  intended 
to  recount  the  findings  from  BTG  and  Global,  and  serve  as  initial  guidelines  to  help  researchers 
determine  how  to  realize  the  full  benefits  of  NCW. 
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